A Modular Approach for Key-Frame Selection in Wide Area Surveillance Video Analysis

This paper presents an efficient preprocessing algorithm for big data analysis. Our proposed key-frame selection method utilizes the statistical differences among subsequent frames to automatically select only the frames that contain the desired contextual information and discard the rest of the insignificant frames. We anticipate that such key frame selection technique will have significant impact on wide area surveillance applications such as automatic object detection and recognition in aerial imagery. Three real-world datasets are used for evaluation and testing and the observed results are encouraging

Deep-Learning-Incorporated Augmented Reality Application for Engineering Lab Training

Deep learning (DL) algorithms have achieved significantly high performance in object detection tasks. At the same time, augmented reality (AR) techniques are transforming the ways that we work and connect with people. With the increasing popularity of online and hybrid learning, we propose a new framework for improving students’ learning experiences with electrical engineering lab equipment by incorporating the abovementioned technologies. The DL powered automatic object detection component integrated into the AR application is designed to recognize equipment such as multimeter, oscilloscope, wave generator, and power supply. A deep neural network model, namely MobileNet-SSD v2, is implemented for equipment detection using TensorFlow’s object detection API. When a piece of equipment is detected, the corresponding AR-based tutorial will be displayed on the screen. The mean average precision (mAP) of the developed equipment detection model is 81.4%, while the average recall of the model is 85.3%. Furthermore, to demonstrate practical application of the proposed framework, we develop a multimeter tutorial where virtual models are superimposed on real multimeters. The tutorial includes images and web links as well to help users learn more effectively. The Unity3D game engine is used as the primary development tool for this tutorial to integrate DL and AR frameworks and create immersive scenarios. The proposed framework can be a useful foundation for AR and machine-learning-based frameworks for industrial and educational training

Multiple Object Detection in Hyperspectral Imagery Using Spectral Fringe-Adjusted Joint Transform Correlator

Hyperspectral imaging (HSI) sensors provide plenty of spectral information to uniquely identify materials by their reflectance spectra, and this information has been effectively used for object detection and identification applications. Joint transform correlation (JTC) based object detection techniques in HSI have been proposed in the literatures, such as spectral fringe-adjusted joint transform correlation (SFJTC) and with its several improvements. However, to our knowledge, the SFJTC based techniques were designed to detect only similar patterns in hyperspectral data cube and not for dissimilar patterns. Thus, in this paper, a new deterministic object detection approach using SFJTC is proposed to perform multiple dissimilar target detection in hyperspectral imagery. In this technique, input spectral signatures from a given hyperspectral image data cube are correlated with the multiple reference signatures using the classassociative technique. To achieve better correlation output, the concept of SFJTC and the modified Fourier-plane image subtraction technique are incorporated in the multiple target detection processes. The output of this technique provides sharp and high correlation peaks for a match and negligible or no correlation peaks for a mismatch. Test results using real-life hyperspectral data cube show that the proposed algorithm can successfully detect multiple dissimilar patterns with high discrimination

Intrusion Detection in Aerial Imagery for Protecting Pipeline Infrastructure

We present an automated mechanism that can detect and issue warnings of machinery threat such as the presence of construction vehicles on pipeline right-of-way. The proposed scheme models the human visual perception concepts to extract fine details of objects by utilizing the corners and gradient histogram information in pyramid levels. Two real-world aerial image datasets are used for testing and evaluation

Multiclass Object Detection with Single Query in Hyperspectral Imagery Using Class-Associative Spectral Fringe-Adjusted Joint Transform Correlation

We present a deterministic object detection algorithm capable of detecting multiclass objects in hyperspectral imagery (HSI) without any training or preprocessing. The proposed method, which is named class-associative spectral fringe-adjusted joint transform correlation (CSFJTC), is based on joint transform correlation (JTC) between object and nonobject spectral signatures to search for a similar match, which only requires one query (training-free) from the object\u27s spectral signature. Our method utilizes class-associative filtering, modified Fourier plane image subtraction, and fringe-adjusted JTC techniques in spectral correlation domain to perform the object detection task. The output of CSFJTC yields a pair of sharp correlation peaks for a matched target and negligible or no correlation peaks for a mismatch. Experimental results, in terms of receiver operating characteristic (ROC) curves and area-under-ROC (AUROC), on three popular real-world hyperspectral data sets demonstrate the superiority of the proposed CSFJTC technique over other well-known hyperspectral object detection approaches

A Robust Fringe-Adjusted Joint Transform Correlator for Efficient Object Detection

The fringe-adjusted joint transform correlation (FJTC) technique has been widely used for real-time optical pattern recognition applications. However, the classical FJTC technique suffers from target distortions due to noise, scale, rotation and illumination variations of the targets in input scenes. Several improvements of the FJTC have been proposed in the literature to accommodate these problems. Some popular techniques such as synthetic discriminant function (SDF) based FJTC was designed to alleviate the problems of scale and rotation variations of the target, whereas wavelet based FJTC has been found to yield better performance for noisy targets in the input scenes. While these techniques integrated with specific features to improve performance of the FJTC, a unified and synergistic approach to equip the FJTC with robust features is yet to be done. Thus, in this paper, a robust FJTC technique based on sequential filtering approach is proposed. The proposed method is developed in such a way that it is insensitive to rotation, scale, noise and illumination variations of the targets. Specifically, local phase (LP) features from monogenic signal is utilized to reduce the effect of background illumination thereby achieving illumination invariance. The SDF is implemented to achieve rotation and scale invariance, whereas the logarithmic fringe-adjusted filter (LFAF) is employed to reduce the noise effect. The proposed technique can be used as a real-time region-of-interest detector in wide-area surveillance for automatic object detection. The feasibility of the proposed technique has been tested on aerial imagery and has observed promising performance in detection accuracy

U-Net and its variants for medical image segmentation: theory and applications

U-net is an image segmentation technique developed primarily for medical image analysis that can precisely segment images using a scarce amount of training data. These traits provide U-net with a very high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in all major image modalities from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. As the potential of U-net is still increasing, in this review we look at the various developments that have been made in the U-net architecture and provide observations on recent trends. We examine the various innovations that have been made in deep learning and discuss how these tools facilitate U-net. Furthermore, we look at image modalities and application areas where U-net has been applied.Comment: 42 pages, in IEEE Acces

Efficient Thermal Image Segmentation through Integration of Nonlinear Enhancement with Unsupervised Active Contour Model

Thermal images are exploited in many areas of pattern recognition applications. Infrared thermal image segmentation can be used for object detection by extracting regions of abnormal temperatures. However, the lack of texture and color information, low signal-to-noise ratio, and blurring effect of thermal images make segmenting infrared heat patterns a challenging task. Furthermore, many segmentation methods that are used in visible imagery may not be suitable for segmenting thermal imagery mainly due to their dissimilar intensity distributions. Thus, a new method is proposed to improve the performance of image segmentation in thermal imagery. The proposed scheme efficiently utilizes nonlinear intensity enhancement technique and Unsupervised Active Contour Models (UACM). The nonlinear intensity enhancement improves visual quality by combining dynamic range compression and contrast enhancement, while the UACM incorporates active contour evolutional function and neural networks. The algorithm is tested on segmenting different objects in thermal images and it is observed that the nonlinear enhancement has significantly improved the segmentation performance

Recent Progress in Wide-Area Surveillance: Protecting Our Pipeline Infrastructure

The pipeline industry has millions of miles of pipes buried along the length and breadth of the country. Since none of the areas through which pipelines run are to be used for other activities, it needs to be monitored so as to know whether the right-of-way (RoW) of the pipeline is encroached upon at any point in time. Rapid advances made in the area of sensor technology have enabled the use of high end video acquisition systems to monitor the RoW of pipelines. The images captured by aerial data acquisition systems are affected by a host of factors that include light sources, camera characteristics, geometric positions and environmental conditions. We present a multistage framework for the analysis of aerial imagery for automatic detection and identification of machinery threats along the pipeline RoW which would be capable of taking into account the constraints that come with aerial imagery such as low resolution, lower frame rate, large variations in illumination, motion blurs, etc. The proposed framework is described from three directions. In the first part of the framework, a method is developed to eliminate regions from imagery that are not considered to be a threat to the pipeline. This method makes use of monogenic phase features into a cascade of pre-trained classifiers to eliminate unwanted regions. The second part of the framework is a part-based object detection model for searching specific targets which are considered as threat objects. The third part of the framework is to assess the severity of the threats to pipelines in terms of computing the geolocation and the temperature information of the threat objects. The proposed scheme is tested on the real-world dataset that were captured along the pipeline RoW